Image distortion transformation method and apparatus

ABSTRACT

An image distortion transformation method for transforming an original image by an imager having an original distortion profile to a transformed image optimized for a distortion processing unit includes inputting the original image from the imager into the transformation unit, inputting an original image distortion profile into the original distortion profile memory of the transformation unit, and inputting a target distortion profile into the target distortion profile memory of the transformation unit. The target distortion profile is different from the original distortion profile. The method further includes transforming the original image into a transformed image by transforming the distortion profile of the original image from the original image distortion profile to the target image distortion profile, and outputting the transformed image from the transformation unit.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional PatentApplication No. 62/295,724, filed Feb. 16, 2016, entitled “ImageDistortion Transformation Method and Apparatus,”, the entire contents ofwhich are incorporated by reference herein.

BACKGROUND OF THE INVENTION

Embodiments of the present invention relate to an image or imagerdistortion processing method and apparatus to transform an image,optimizing it for a distortion processing algorithm (e.g., dewarpingalgorithm). Using the same method, embodiments of the invention allowmagnification of some areas of interest in the image or imager field ofview, optimizing the storage, transmission, resolution, or display ofthe content.

A two-dimensional (2D) image is created by an imager, such as a narrowangle imager, panoramic imager, wide-angle optical system, fisheyeoptical system, panomorph optical system, or even image stitching frommultiple narrow-angle images or image stitching from multiples panoramicimages. No matter what kind of imager is used, the image often containsoptical distortion. This behavior is caused by remapping of thethree-dimensional (3D) environment to a 2D plane. As the result,contents in the image are deformed, and the exact distortion profiledepends on the imager parameters. Usually, the distortion is much moreobvious with panoramic imagers.

To properly display an image without distortion, or feed the correctimage to any image processing algorithm, the distortion in panoramicimage has to be processed, as in U.S. Pat. No. RE 44,087 E for lineardistortion or U.S. Pat. Nos. 6,844,990 B2 and 6,865,028 B2 for customsymmetric and custom asymmetric distortion, respectively. These existingdistortion processing algorithms or dewarping algorithms (actions tocorrect, remove, process or modify the distortion of an image) areimplemented by software or hardware on different platforms, such as acomputer, smartphone, tablet, System on Chip (SoC), or the like. As anexample, IPIX plug-in for Milestone system is a dewarping software tocorrect distortion of fisheye lenses. Some dewarping algorithms use onlya linear distribution function to correct different distortion profilesfrom different panoramic imagers. In this case, because the linearcorrection function used does not match the real distortion profile ofthe panoramic image or imager, some distortion remains in the imagesafter dewarping. To properly correct the distortion, the dewarpingalgorithm has to use the distribution function (distortion profile),considering the distortion profile of the panoramic imager and theapplication user experience. This means that the departure between thedistortion profile and the dewarping algorithm distribution function hasto be controlled. One approach could be to input the distortion profileas a mathematical function to the distortion processing algorithm tocorrect a panoramic image. However this approach cannot be used in allsituations, for example, but not limited to, sometimes the distortionprofile is confidential or not present in the image processing unitrunning the distortion processing algorithm. Furthermore, the distortionprofile for a new imager cannot be easily added or updated in existingsoftware or in a hardware processing algorithm. This requires a softwareupdate, firmware update, or hardware modification (ASIC modification,for example).

ImmerVision Enables 2.0 algorithms correct the distortion of panomorphimages or imagers. The libraries including the algorithms contain somedistortion profiles (or distribution functions) corresponding to thepanomorph imagers or panomorph lenses available at the time of thealgorithm release in the image processing units (library, ASIC, FPGA, orthe like). If the distortion profile of a panoramic image or imager isin this list of pre-recorded distortion profiles, a user only needs toinput a distortion profile reference number and the processing algorithmwill use the proper existing distortion profile. For example, at leastsome embodiments of U.S. Patent Application US 2015/0281507 A1 use amarker on the image to record this distortion profile reference numberor the distortion profile itself. In the embodiments where only thereference number of the distortion profile is marked, the distortionprocessing unit can read the reference number from the marker andautomatically use the corresponding distortion profile. Compared withtransferring the whole distortion profile to the distortion processingalgorithm, this way is more efficient. In some cases, this approachcannot be used since the distortion profile is not present in thealgorithm, and therefore the algorithm is not designed to support thisnew type of distortion profile. An example of this case is a panoramicimage with a new distortion profile which is created by, for example,but not limited to, a new wide-angle, fisheye, or panomorph lens, or newimage processing algorithm. For this image, the distortion processingunit has to be updated with the new distortion profile or a completelynew algorithm before the distortion processing unit can correct, remove,process or modify the distortion of the panoramic image or imager. Insome situations, algorithm updates cannot be done easily, especiallywhen the distortion processing algorithm is implemented on hardware.This limits the application of distortion processing algorithms fordifferent panoramic images or imagers.

Due to the big data size of images or videos, the optimization of imagestorage or transmission resource is always preferred. An optimizationapproach is to reduce image or video size. For panoramic image orwide-angle images, U.S. Pat. No. 8,238,695 proposes to crop a part ofimage data for a virtual camera from the whole image to reduce imagedata size. This solution sacrifices the image data outside of thevirtual camera area, and it can only be used for the use case of avirtual camera. The saved or transmitted image only contains image datafor the virtual camera, and other image data from the rest of the fieldof view is lost. This does not work for the application of wholepanoramic images. Another alternative, to keep the whole panoramic fieldof view, is to instead uniformly scale down the whole image to a lowerresolution and to decrease the image size. However, this scaling is thesame for the whole image, and the image quality is degraded in the wholeimage. One optimized approach would be to use higher resolution inimportant image areas and lower resolution in other image areas, meaningdifferent scaling ratios are applied for different areas of the image.In this way, more pixels would be used to emphasize important areas whenthe image size is decreased. This non-linear scaling creates a newdistortion in the generated image. This new distortion is different fromthe original distortion of the original panoramic image. So thedistortion in the original panoramic image cannot be used for processingthis optimized image.

U.S. Pat. No. 6,865,028 proposes to project non-disk shape panoramicimage on a rectangular image sensor. This includes, but is not limitedto, elliptical images. Compared with a circular image, the ellipticalimage occupies more pixels on the rectangular image sensor and there aretherefore fewer black pixels in the output image. In this way, out ofthe total file size for the image, a larger fraction is used by usefulimage contents compared to useless black content. The resulting imagehas higher effective resolution. For image storage and transmission,this elliptical panoramic image containing fewer black pixels is moreefficient than a circular panoramic image. However, U.S. Pat. No.6,865,028 only suggests optical means to implement this method ofcreating panomorph optical system, and the distortion profile is uniquefor each panomorph optical system. In addition, the distortion profileis fixed after the panomorph optical system is produced. The distortionprofile can only be modified by designing new lenses, so an existingoptical lens cannot be modified and cannot dynamically change thedistortion depending on applications.

Dynamically changing the distortion has been proposed in the past inU.S. Patent Publication No. 2012/0314065 A1. However, this inventionchanges the distortion in real-time using a hardware optical systemconsisting of a deformable optical surface as a deformable mirror. Thisconstruction is complex and cannot be used in existing panoramic imagersto dynamically modify the areas of interest.

Since 2006, the Panomorph Geometrical Model Simulator (PGMS) software,developed by ImmerVision, simulates distortions in images to analyzeimage distortion effects. The Panomorph Geometrical Model was describedin the paper “Novel hemispheric image formation: concepts andapplications” published in “Proc. SPIE 6994, Photon Management III,699406 (Apr. 25, 2008)”. The software can create different distortionson images, including different pre-set distortions, non-linear scalingdistortions, or any customized distortions. However, the software is notfor panoramic image distortion processing, and it cannot transform apanoramic image with a new distortion profile to be optimized fordistortion processing algorithms. It cannot transfer panoramic imagedistortion to a non-linear scaling distortion to emphasize importantareas in an image either. Finally, it cannot be integrated into apanoramic image processing pipeline.

BRIEF SUMMARY OF THE INVENTION

To overcome all the previously mentioned issues, embodiments of thecurrent invention propose an image distortion transformation method totransform images captured by different imagers to be optimized fordistortion processing algorithm. This allows an image distortionprocessing algorithm to process images captured by different imagers.Here, the imagers include, but are not limited to, narrow-angle lenses,wide-angle lenses, fisheye lenses, panomorph lenses, multiple-lenssystems, catadioptric systems, panoramic images generators by imagestitching or any image processing units, or the like. The imagedistortion processing algorithm is used to process image distortion forimage display or any further image processing algorithm. The distortionprocessing algorithm can be implemented by software or hardware ondifferent platforms, including, but not limited to, a computer,smartphone, tablet, Field-Programmable Gate Array (FPGA),Application-Specific Integrated Circuit (ASIC), System on Chip (SoC), orthe like. The image distortion transformation method is used totransform the original image to an image with any kind of distortionprofile already supported by the image distortion processing unit. Thisimage distortion transformation method can be implemented by software orhardware on different platforms, including, but not limited to, acomputer, smartphone, tablet, FPGA, ASIC, SoC, or the like. This methodalso can be implemented by an optical system or any kind of system orassociated devices. This image distortion transformation method can beimplemented at different places, including, but not limited to, cameras,image capture devices, media servers, client computers, display devices,image processing algorithms, or a cloud system.

More particularly, in at least some embodiments of the presentinvention, a panoramic original image captured by a panoramic imager istransformed to an image with a well-known distortion profile, which isalready supported by the distortion processing algorithm. The distortionprocessing algorithm can process distortion according to this supporteddistortion profile.

To optimize the image storage, transmission, resolution, quality ordisplay, an efficient approach is to allocate more pixels to areas ofinterest in the image. More particularly, in at least some embodimentsof the present invention, a target distortion profile is designed toincrease the number of pixels occupied by an area of interest anddecrease the number of pixels occupied by less important contents. Theareas of interest have higher resolution (or pixel density) than lessimportant contents. The distortion transformation method can transformthe original image to a transformed image with an appropriate profilebefore the image is saved, transmitted, analyzed, processed, ordisplayed. The distortion transformation method can vary themagnification across the image and is not a linear scale of the originalimage. In this way, the important image areas are emphasized while theimage size may be kept the same. In some embodiments, this invention canbe used to scale down the image while keeping more pixels in areas ofinterest than the other part of the image. In some other embodiments,this transformation can be done dynamically according to the applicationrequirements in real-time.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The foregoing summary, as well as the following detailed description ofa preferred embodiment of the invention, will be better understood whenread in conjunction with the appended drawings. For the purpose ofillustration, there is shown in the drawings an embodiment which ispresently preferred. It should be understood, however, that theinvention is not limited to the precise arrangements andinstrumentalities shown.

In the drawings:

FIG. 1 is a schematic of the image transformation process to transforman original panoramic image to a transformed panoramic image with atarget distortion profile known to the distortion processing unit;

FIG. 2 is a schematic detailing the image transformation method insidethe image transformation unit;

FIG. 3 shows multiple examples of original image formats and multipleexamples of transformed images;

FIG. 4 is an example embodiment of the present invention where theselection of the transformed image distortion profile is made accordingto some parameters of the original panoramic imager;

FIG. 5 is an example embodiment of the present invention where theselection of the transformed image distortion profile is changeddynamically according to variations in the image content of the originalpanoramic image;

FIG. 6 is an example embodiment of the present invention where the imagetransformation unit generate multiples transformed images from anoriginal panoramic image, each having a different distortion profile fora given application;

FIG. 7 is an example embodiment of the present invention where the imagetransformation unit generate multiples transformed images from anoriginal panoramic image, each having a different target distortionprofile inputted from different devices having different applications;and

FIG. 8 is an example embodiment of the present invention where the imagetransformation unit generates a transformed image from an originalpanoramic image, the transformed image changing in real-time the targetdistortion profile inputted from the display device depending on thelocation of the areas of interest needed for its application.

DETAILED DESCRIPTION OF THE INVENTION

The words “a” and “an”, as used in the claims and in the correspondingportions of the specification, mean “at least one.”

FIG. 1 shows a schematic of the image distortion transformation methodto transform a panoramic image from a panoramic imager to an image withtarget distortion profile. A capturing device 100 is preferably builtwith a panoramic imager 110. Alternatively, the capturing device 100could be built with one or more narrow-angle imagers 105. The capturingdevice 100 may be any apparatus on which a panoramic imager could beadded to add a panoramic imaging feature, including for example but inno way limited to, a mobile phone, tablet, car, robot, TV, router, GPSunit, Internet box, home surveillance camera, security camera,automotive camera, endoscope, or the like. When the panoramic imager 110is integrated into the capturing device 100, this device is able tocapture an original panoramic image 120.

Here, the panoramic imager 110 is any device capable of capturing imagesor parts of images from a wide-angle panoramic environment. Theresulting panoramic environment preferably produces a field of view ofabout 120° or larger. Some examples of a panoramic imager 110 that canbe used in the process are, but in no way limited to, wide angle lenses,catadioptric imagers, multiple narrow field of view cameras, computergenerated panoramic images, and the like.

The original panoramic image 120 is the output image of capturingdevices 100. The panoramic image 120 contains panoramic image contents125. There is some original image distortion in the panoramic imagecontents 125. When the panoramic image distortion processing unit 160does not know the distortion profile in the original panoramic image120, the image distortion cannot be readily processed. Embodiments ofthe present invention propose to add an image distortion transformationunit 130 to transform the original panoramic image 120 to an image witha target distortion profile, which is known by the distortion processingunit. In this way, the distortion in the original panoramic image 120can be optimized for the distortion processing unit 160.

Both the original panoramic image 120 and transformed panoramic image140 can be any kind of panoramic image, including, but not limited to,circular or elliptical projected image, perimeter projected image,non-linear scaled image, wide-angle image with continuous,discontinuous, symmetric, or asymmetric or any other projection.

The original panoramic image 120 is the input image of image distortiontransformation unit 130. The image distortion transformation unit 130can be implemented by software or hardware. It can be implemented on anyplatform, including but not limited to, computer, smartphone, tablet,FPGA, ASIC, SoC, or the like. It can also be implemented by opticalsystems or any kind of systems or associated devices. This method can beimplemented at different places, including, but not limited to, cameras,image capture devices, media servers, client computers, panoramicdisplay devices, panoramic image processing algorithms, or cloudsystems. In a way that will be further explained with respect to FIG. 2,the image distortion transformation unit 130 outputs transformedpanoramic image 140. The transformed panoramic image 140 contains atarget distortion profile.

More particularly, in at least some embodiments of the presentinvention, there exist some public distortion profiles for panomorphimages, a kind of panoramic image created by panomorph imager whichcreates a zone of interest. The panomorph distortion processing softwareor hardware contain these public distortion profiles and can process anyimage mapped using these public distortion profiles. The distortiontransformation method proposed in this invention transforms the originalpanoramic image to a transformed image with one of the public panomorphdistortion profiles. In this way, panomorph distortion processingsoftware or hardware can process distortion of any original panoramicimages after the distortion transformation unit.

Before displaying or processing in an algorithm unit the panoramicimage, the distortion in a transformed panoramic image 140 needs to beprocessed by a distortion processing unit 160. The transformed panoramicimage 140 can be directly inputted to algorithm unit 175 or todistortion processing unit 160. Alternatively, the transformed panoramicimage 140 can be outputted through image storage or differenttransmission ways 150, including, but in no way limited to, transfer ofdigital images over the Internet, transfer via TCP-IP protocol, USB,Firewire, memory, socket, transfer using a physical medium as a floppydisk, a laser disk, a magneto disk, a USB drive, a hard drive, ortransfer of the image through physically printing the image and thenscanning it back into a digital file, and the like. When the image istransmitted, it can be done internally inside a device or externally toanother device.

Because the distortion processing unit 160 knows the target distortionprofile in the transformed panoramic image 140, it can process thedistortion and produce processed images 180.

The processed image 180 can be displayed on display devices 170. Thedisplay devices 170 can be anything displaying in part or in full theresult of the distortion processing unit, including but in no waylimited to, a computer or TV screen, a mobile phone, a tablet, a virtualreality (VR) headset, an augmented reality (AR) headset, a projector, orthe like. In addition, the processed image 180 or the transformedpanoramic image 140 can also be used inside a further algorithm unit 175that further processes the processed image to extract relevantinformation, including, but in no way limited to, pedestrian detection,tracking and recognition, face detection, road lane departure, dangeridentification, 3D position tracking, 3D reconstruction, or the like.

In at least some embodiments of the present invention, the displaydevices 170 or the algorithm unit 175 can communicate via an optionallink 190 with the image distortion transformation unit 130 and changethe targeted distortion profile of the transformation unit in real-timedepending the requirements of its application.

FIG. 2 shows the detailed operations of the image distortiontransformation unit 130 in FIG. 1. The image distortion transformationmethod 230 is implemented inside the image distortion transformationunit 210. In addition, the image distortion transformation unit 210 alsocontains the original distortion profile memory 220 and the targetdistortion profile memory 225. The original image distortion profile 250and the target image distortion profile 260 are saved in these twomemory units respectively. The image distortion transformation method230 makes use of original image distortion profile and target imagedistortion profile to transform the original panoramic image 200 to atransformed panoramic image 240 which is distorted according to thetarget distortion profile. The image distortion transformation method230 can include, but is not limited to, pixel mapping, interpolation,filtering, or different image processing algorithms.

The original panoramic image 200 is the input image of the imagedistortion transformation method 230. Each different original panoramicimage 200 can contain a different original image distortion profile 250,which is created by different panoramic imagers. The image distortiontransformation method 230 needs to know the distortion profile of theoriginal panoramic image 200. The original image distortion profile 250can be inputted to the original distortion profile memory 220 from someinternal or external sources, including but not limited to, a manualselection from a list of pre-saved image distortion profiles from a userinterface, an automatic analysis of the content of the original imageincluding reading a marker or metadata in the original image, orautomatically transmitted from the imager. In these cases, the originalimage distortion profile 250 is written inside the original distortionprofile memory 220. In other embodiments of the present invention, as isthe case with an ASIC used for a specific transformation application,the original distortion profile can be hard-written inside the originaldistortion profile memory 220 and cannot be adjusted.

The image distortion transformation method 230 needs to know the targetimage distortion profile of the output transformed panoramic image 240,which can be processed by existing distortion processing algorithms. Thetarget distortion profile could be, for example, but in no way limitedto, one of the several public panomorph distortion profiles. In thiscase, the target image distortion profile 260 can be pre-saved in thetarget distortion profile memory 225, as is the case with an ASICprogrammed for a single specific application. In addition, the targetimage distortion profile 260 can also be inputted to target distortionprofile memory 225 from some internal or external sources. The targetdistortion profile 260 can be a selection from a list of pre-savedtarget image distortion profiles inside the target distortion profilememory 225 or a whole image distortion profile. The sources of targetimage distortion profile 260 include, but are not limited to, an imagedistortion processing unit, panoramic image display devices, facetracking or other image processing algorithm or the like. The imagedistortion transformation method 230 transforms the original image tothe target image with a distortion profile optimized for existingdistortion processing algorithms. The output transformed panoramic image240 with the target distortion profile can later be processed byexisting distortion processing algorithms. As a result, without anymodification or updating to the distortion processing unit, thedistortion processing unit can process any panoramic images after theimage distortion transformation method 230.

Both the original image distortion profile 250 and the target imagedistortion profile 260 can be static distortion profiles. In someembodiments of the present invention, they also can be dynamicdistortion profiles. Original image distortion profile 250 can be, forexample, but in no way limited to, updated in real-time when thedistortion profile of the original panoramic image 200 changes with timefor dynamic panoramic imager. In other embodiments, target imagedistortion profile 260 can be, but in no way limited to, updated todifferent target image profiles required by application.

Several examples of original images and transformed images are shown atFIG. 3. The original panoramic image of image distortion transformationmethod 300 can be a circular panoramic image 310 or an ellipticalpanoramic image 317. In some others embodiments of the presentinvention, the image distortion transformation method 300 takes originalmultiple images from multiple narrow images 320 or original multiplesimages from multiple panoramic images as input and stitches them into asingle or dual panoramic image with a target transformed distortionprofile. In some other embodiments of the present invention, the inputcan also be an original equi-rectangular image 330 or a single narrowimage 335.

Through image distortion transformation method 300, the outputtransformed panoramic image can have different target distortions. Someexamples of output transformed panoramic images are as 350, 360, 365,370, 375, 380, 385 and 390. After transformation, more pixels ofinformation are recorded inside areas of interest. In cases where thetotal image size is reduced, the present invention allows retention ofall the original information in the areas of interest.

As a first example, some fisheye or panomorph imagers create images withoriginal panoramic image contents 315 only in a central disk area, suchas in original panoramic image 310. The image is black outside theoriginal panoramic image content disk. With rectangular image files,which are outputted by an image sensor with 16:9 or 4:3 aspect ratio,the black boundaries are even larger compared to the original imagecontent disk. In this kind of image, the black boundaries are notuseful. When the black boundaries are narrowed and occupy fewer pixels,the useful image contents will occupy more pixels and have higherresolution.

A simple distortion profile transformation is to stretch the circulararea with image contents to an elliptical area, as in transformedpanoramic image A 350. Compared with original panoramic image contents315, transformed image contents 355 occupy more pixels in the image andthere are fewer black background pixels. More sophisticated distortionprofiles can also be applied, including, but not limited to,super-elliptical shapes, as in transformed panoramic image B 360, ornon-linear planispheric projections, as in transformed panoramic image H390. When a panoramic image is transformed in this way before it isstored or transmitted, more image storage or transmission resource canbe used on important contents instead of useless black content.

As another example, the image display software or hardware can requirethe image distortion transformation method 300 to increase resolution ofan area of interest when a user or device wants to zoom in the area.Compared with the digital zoom on a degraded image after image storageor transmission, better image quality can be achieved when resolution isincreased by the image distortion transformation method 300 before imagestorage or transmission. The image distortion transformation method 300can increase resolution at some positions, as in transformed panoramicimage C 365 with edge magnification, transformed panoramic image D 370with central magnification, and transformed panoramic image E 375 withmid-zone magnification. The exact position of the area of interest ischosen according to the requirement from a user or an application.

As another example, the image distortion processing algorithm isimplemented by hardware. With this specific hardware, only circularsymmetric distortion profiles are supported. When the original panoramicimage is without rotational symmetry as in original ellipse panoramicimage 317, the image distortion transformation method can transform theimage to a transformed image with rotationally symmetric distortionprofile.

As another example, the target distortion profile could be any kind ofprojection distortion which can be processed by image distortionprocessing unit. The transformed panoramic image could be ellipticalpanoramic image 350, super-elliptical 360, circular panoramic image 365,370 and 375, or parametric projected panoramic image 390. In mostembodiments according to the current invention, the field of view of theoriginal image and the field of view of the transformed image aresimilar. Alternatively, the transformed panoramic image could also be adewarped part of the full image in areas of interest having a smallerfield of view as in transformed image F 380. More particularly, the caseof transformed image F at 380 shows that in some embodiments of thecurrent invention, keeping the full panoramic view by compressing therest of the image is not mandatory and we could combine this processingthat creates a zone of increased resolution with a cropping of theother, less important, part of the image if required by the application.

As another example, when the input image is original narrow image 335with a specific distortion profile, the output transformed image G 385can also be narrow angle with a different distortion creating an area ofinterest.

As another example, when the input image is original multiple images 325as in the case of stereoscopic panoramic imaging, the output can betransformed multiple image F 380 to allow displaying a differenttransformed image to each eye after processing while increasing thenumber of pixels in the area of interest.

FIG. 4 shows an example embodiment of the present invention where theselection of the output transformed image distortion profile is madeaccording to some parameters of the original panoramic imager. In atleast some embodiments of the present invention, the target distortionprofile of the image distortion transformation process from FIG. 3 canbe a dynamic distortion profile required by the application and thetarget distortion profile can be changed in real-time according to anyparameter from the original image or the imager. These parametersinclude, but are in no way limited to, device orientation, deviceposition or altitude from a GPS device or the like, date and time,surrounding environmental conditions including weather or imagingunderwater or user selection. As one specific example, in no waylimiting the scope of the invention, the distortion transformationmethod can use different target distortion profiles for differentorientations of the capturing device. When the panoramic imager 400faces down or up, the image contents on the boundary are more importantbecause they are the objects around camera. In this case, the imagedistortion transformation unit 420 can transform the original panoramicimage to transformed panoramic image 430 with higher resolution towardthe edges of the image content. Alternatively, when the panoramic imager410 faces front, the image distortion transformation unit 420 cantransform the original panoramic image to the transformed panoramicimage 440 with higher resolution in the central area, which containsobjects in front of camera.

FIG. 5 shows another example embodiment of the present invention wherethe target distortion profile is dynamically adjusted according to thescene content. The adjustment of the distortion profile can be from anysource, including a user manually moving the area of interest or analgorithm unit doing specific object recognition or face tracking. Inthe example shown, there is a person moving in the field of view of thepanoramic imager and the area of interest follows him. At first, theperson 502 is on the left of the object scene 500 imaged by thepanoramic imager 530. The image distortion transformation unit 540 isconsequently configured to create an area of interest with highermagnification 552 in the left part of the image 550, while still keepingthe full panoramic view by compressing the rest of the image. At a latertime, the person 512 is now in the center of the object scene 510 imagedby the same panoramic imager 530. The image distortion transformationunit 540 is consequently configured to create an area of interest withhigher magnification 562 in the central part of the image 560, whilestill keeping the full panoramic view by compressing the rest of theimage. At an even later time, the person 522 is on the right of theobject scene 520 imaged by the same panoramic imager 530. The imagedistortion transformation unit 540 is consequently configured to createan area of interest with higher magnification 572 in the left part ofthe image 570, while still keeping the full panoramic view bycompressing the rest of the image.

FIG. 6 shows an example embodiment of the present invention where theimage distortion transformation unit 605 outputs multiple transformedimages from the same original panoramic image 600. Alternatively, inanother embodiment according to the present invention, the multipletransformed images could also originate from multiple original panoramicimages as in the case of stereoscopic vision. In this specific example,in no way limiting the scope of the present invention, there are twotransformed images 610 and 615, each having a different area ofinterest. The image on the left 610 has more pixels in the area ofinterest in the center of the image while the image on the right 615 hasmore pixels in the area of interest at the right of the image. Thesemultiple transformed images are then stored or transmitted at 620 to beused by the image distortion processing unit 630. When the image istransmitted, it can be done internally inside a device or externally toanother device. After distortion processing, two processed images 640and 645 are outputted, each image showing the part of the image wherethe area of interest was in the transformed images 610 and 615. Theseprocessed images 640 and 645 are then used inside a single or multipleviewing devices 650. These devices could be, for example, a virtualreality viewing apparatus. In at least some embodiments of the presentinvention, there is also an optional communication link 660 between theviewing device 650 and the image distortion transformation unit 605,allowing the viewing device to modify, in real-time, the position andsize of the area of interest according to the required application.

FIG. 7 shows an example embodiment of the present invention where theimage distortion transformation unit 705 outputs multiple transformedimages 710 from the same original panoramic image 700. The output imagesmay have any output transformed distortion according to the presentinvention. The target transformed image distortion profile is configuredaccording to the requirements of the application and transmitted to theimage distortion transformation unit 705 via the communication link 750.The transformed images 710 are then stored internally or transmitted at720 to a distortion processing unit 730. When the image is transmitted,it can be done internally inside a device or externally to anotherdevice. The output from this distortion processing unit 730 is themultiple display devices 740. According to their application, thedisplay devices 740 request, in real-time, the exact transformeddistortion images to the image distortion transformation unit 700 viathe communication link 750. Each different display device can have adifferent area of interest or a different transformed image distortionprofile.

FIG. 8 shows an example embodiment of the present invention where theimage distortion transformation unit 805 continuously generatestransformed panoramic images 810 from an original panoramic image 800and outputs them to an image storage or transmission 820 to be used in adistortion processing unit 830. When the image is transmitted, it can bedone internally inside a device or externally to another device. Thedistortion processing unit 830 receives a transformed image containingthe full panoramic content, or a part of the panoramic content withlarger field of view than what is required to be displayed. Thedistortion processing unit 830 extracts and dewarps a region of theimage, centered on the areas of interest of the panoramic image,requested by the display devices 840. These display devices 840 displaythe dewarped view of the image centered on the areas of interest. Whenthe display devices require a new area of interest, as for example therotation of the head of a user using a virtual reality display device,the display devices 840 communicate with the image distortiontransformation unit 810, via the communication link 850, in order tohave a new transformed panoramic image 810 with this new area ofinterest. This process can take some time because of latency between thedisplay devices and the image distortion transformation unit. However,thanks to the present invention creating transformed images still havingthe full panoramic content in the images, or a part of the panoramiccontent with larger field of view than what is required to be displayed,with lower resolution outside the areas of interest, the distortionprocessing unit 830 can instantaneously dewarp the content of the newarea of interest and display the result in the display devices with alower resolution. After this latency period is over, the distortionprocessing unit finally receive the transformed images with the newtarget distortion profile having the maximum resolution in the new areaof interest. The resulting dewarped view displayed in the displaydevices 840 is improved from the lower resolution view to the maximumresolution view. Furthermore, in some embodiments of the presentinvention, only the difference between two transformed images from theimage distortion transformation unit 805 are transmitted or stored,optimizing the storage or transmission and reducing the latency of thesystem.

All of the above are figures and examples of specific image distortiontransformation units and methods. In all these examples, the imager canhave any field of view, from very narrow to extremely wide-angle. Theseexamples are not intended to be an exhaustive list or to limit the scopeand spirit of the present invention. It will be appreciated by thoseskilled in the art that changes could be made to the embodimentsdescribed above without departing from the broad inventive conceptthereof. It is understood, therefore, that this invention is not limitedto the particular embodiments disclosed, but it is intended to covermodifications within the spirit and scope of the present invention asdefined by the appended claims.

What is claimed is:
 1. An image distortion transformation method for transforming an original image by an imager having an original distortion profile to a transformed image optimized for a distortion processing unit, the method being used for creating an area of interest in the transformed image, the method comprising: a) inputting the original image from the imager into the transformation unit; b) inputting an original image distortion profile into the original distortion profile memory of the transformation unit; c) inputting a target distortion profile into the target distortion profile memory of the transformation unit, the target distortion profile being different from the original distortion profile; d) transforming the original image into a transformed image by transforming the distortion profile of the original image from the original image distortion profile to the target image distortion profile, the transformed image not being a linear scale of the original image; and e) outputting the transformed image from the transformation unit.
 2. The image distortion transformation method of claim 1, wherein the area of interest in the original image is adjusted dynamically according to moving content inside the original image.
 3. The image distortion transformation method from claim 1, wherein the original image from the imager is a panoramic image.
 4. The image distortion transformation method from claim 1, wherein the original image and the transformed image have a similar field of view.
 5. The image distortion transformation method from claim 1, wherein the original image from the imager is composed of multiple images.
 6. The image distortion transformation method of claim 1, wherein the target distortion profile is selected by the transformation unit according to a parameter of the imager.
 7. The image distortion transformation of claim 6, wherein the parameter is the orientation of the imager.
 8. The image transformation method of claim 1, wherein the transformed image is outputted to a processing unit.
 9. The image transformation method of claim 8, wherein the target distortion profile is supported by the processing unit.
 10. The image distortion transformation method of claim 8, wherein the distortion profile is transmitted from the transformation unit to the processing unit or the distortion profile is transmitted from the processing unit to the transformation unit.
 11. The image distortion transformation method of claim 10, wherein the distortion profile is transmitted with the transformed image.
 12. The image distortion transformation method of claim 1, further comprising a display device, wherein the distortion profile is transmitted from the display device to the transformation unit.
 13. The image distortion transformation method from claim 1, wherein the transformed image is outputted to a display device, the method further comprising: f) adjusting, by the display device, the inputted target distortion profile of the transformation method according to a specific application required by the display device.
 14. The method of claim 13, wherein the inputted target distortion profile of the transformation method is adjusted to modify the position, size or magnification of an area of interest.
 15. The method of claim 1 wherein the original distortion profile is inputted in the transformation unit either from a manual selection through a user interface, automatically from analyzing the content of the original image, automatically transmitted directly from the imager or hard-coded inside the original distortion profile memory.
 16. The method of claim 1 wherein the target distortion profile is inputted in the transformation unit either from a manual selection through a user interface, automatically transmitted from either the distortion processing unit, the display device or an algorithm unit or hard-coded inside the target distortion profile memory.
 17. An image distortion transformation method inside an image distortion transformation unit used in combination with a distortion processing unit, the method being used for creating an area of interest in the transformed image, the method comprising: a) a distortion processing unit configured to process a distortion profile; b) an image distortion transformation unit transforming an original image to a transformed image according to said distortion profile, the transformed image not being a linear scale of the original image.
 18. An image capturing system for transforming an original image by an imager having an original distortion profile to a transformed image optimized for a distortion processing unit, the system being used for creating an area of interest in the transformed image, the system comprising a transformation unit having an original distortion profile memory and a target distortion profile memory, the transformation unit being configured to: a) receive the original image from the imager; b) input an original image distortion profile into the original distortion profile memory; c) input a target distortion profile into the target distortion profile memory, the target distortion profile being different from the original distortion profile; d) transform the original image into a transformed image by transforming the distortion profile of the original image from the original image distortion profile to the target image distortion profile, the transformed image not being a linear scale of the original image; and e) output the transformed image. 